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The Peer Eval'iation Program (PEP) has been instituted 
by the Division of Special Education, Pennsylvania State Department* 
of Education, to allow intermediate units an opportunity to 
continuously improve their programs. The advantages of the PEP system 
are: (1) it is a self-im prove memt system of program development; (2) 
PEP allows local autonomy as to what changes are to occur; (3) PEP 
builds into the evaluation structure a deliberate mechanism for the 
change process and for implementing strategies derived; (4) feedback 
is meant to be formatively specific rather than descriptively 
general; (5) PEP serves valuable in-service and consultation 
function; (6) there is participation of all staff members at the 
local level; and (7) many changes can be identified that require no 
increase in resources. General procedures of the program are 
outlined; these relate to: Pre- Visit Preparations (internal 
evaluation, selection of evaluators, requested time lines, in-service 
programs, internal reports, and selection of evaluation team 
members) ; Arrangement for On-site Visits; On-site Visits (arrival f 
procedures, and post-visit feedback) ; Change Mechanism for 
Implementing Recommendations (local team, final disposition of 
"change strategies," implementation and monitoring of board 
resolutions) • The most important single component of the PEP is the 
selection of the evaluation team; criteria for selecting the members 
are provided* Two evaluation forms are included: "Teacher Input for 
Self Evaluation" and "Criteria for Use in Both Self- and 
Peer-Evaluations-" A flowchart of the PEP is provided, (DB) 
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ERIC Clearinghouse on Tests, 

Measurement and Eva] uat i on 
Educational Testing Service 
Princeton, New Jersey 08540 

Gentlemen: 

We are enclosing two (2) copies of a program evaluation 
document entitled, "Guidelines: 'PEP', Peer Evaluation Programs: 
A Systematic Approach for Evaluating Educational Programs." The 
document contains an innovative approach to statewide evaluation 
that is currently being implemented in the Commonwealth of Penn- 
sylvania. We think others at the national, state, and local levels 
would also be interested. Thus, we nope the document can be made 
available through Research in Education ( R I E) and ERIC microfiche. 

l-f you consider the document appropriate for inclusion in 
ERIC, the following mechanical details might be of some help in 
cataloging and indexing. /The date of publication is August, 1973, 
and the authors should read Jeffrey N. Grotsky, Barton B. Proger, 
Robert C. Warkomski , David L. Hayden, William Murphy, John A. 
V*y Abbruzzese, Jr., and Earl Bonnett . j Wh i 1 e. we do not have an ERIC 

y Thesaurus before us, I would suggest descriptors as follows: 
xfy Program Evaluation, Accountability, Educational Management, In- 
t> structional Evaluation, Needs Assessment, Formative Evaluation, 
Accreditation, and Systems Analysis. 

Please notify me of your decision to include or reject the 
enclosed document for your ERIC system. Thank > ou for your 
attention to this matter. 



S i ncerel y , 

Barton B. Proger, Ed. D 
Coordinator of Evaluation Services 
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PHILOSOPHY AND RATIONALE 

Change for the sake of program improvement in Special Education has 
usually been left to more or less informal processes. Small day-to-day 
changes are always made by teachers, psychologists, administrators and 
so on. However, such changes typically do not have a wide-range effect 
on the total program structure and operations. In line with notions of 
accountability, the Peer Evaluation Program (PEP) has been instituted by 
the Division of Special Education to allow intermediate units an oppor- 
tunity to improve their programs continuously. There are many advantages 
to the PEP system as outlined in ,? General Procedures". 

First, PEP is to be considered a self -improvement system of program 
development. It is a system of peer-help and self-help done in a mutual 
give and take professional fashion, Individual staff members are not under 
scrutiny, The intermediate unit program as a whole is the only thing being 
evaluated for purposes of common^sense improvement. The ultimate goal is 
to make system-wide changes in intermediate unit programs rather than the 
usual, isolated improvements that occur without the knowledge of other staff 
members . 

Second, PEP allows local autonomy as in the matter of what changes 
are to occur. After both the self -evaluation and on-site peer evaluations 
have occurred, it is left to local IU ataff as to what changes are feasible 
and appropriate at the given time (compatible with local constraints and 
context), which changes will be carried out and how they will be implemented. 
Final approval of the changes anticipated and amendments to IU plan remain 
in the hands of the Division of Special Education as the approving body of 



IU Special Education plans. Thus, the changes to be attempted are self- 
initiated and not forced upon the IU, However, it should be noted that 
a basic assumption of the PEP system is that if an IU is not attempting 
major changes in program structure, philosophy, curriculum, etc., then some- 
thing is askew. 

Third PEP builds into the evaluation structure a deliberate mechanism 
for the change process and for implementing strategies derived therefrom. 
It has been demonstrated that formal program evaluations that go only so 
far as to submit a written report are not very effective. In this sense, 
the vast majority of program evaluation efforts have not achieved the de- 
sired change. Administrators charged with program control and improvement 
might have been able to read through the report and perhaps have done some 
serious thinking about changes that are recommended % However, because of 
lack of time and lack of specific recommendations, usually no changes of 
real consequence ever occurred. Thus, evaluation reports usually gathered 
dust till the next program evaluation session. In comparison, PEP struc- 
tures a Change Committee to consider where legitimate problem areas are 
on the basis of both self- and peer^evaluation feedback, what alternatives 
exist to overcom ;he deficiency, what the pros and cons of each alternative 
are with regard to effects on the total system, and choosing a final change 
strategy and means of implementing it* 

Fourth, the PEP feedback is meant to be formatively specific rather 
than descriptively general. While it would have been possible to use 
accreditation-type checklists (such as Middle States criteria or the Penn- 
sylvania Standards for the Operation of Special Education Programs and Services) , 
these are only minimal standards that tell what a program should or should 
not have but not how to correct it, Thus, only a general description is 



provided by accreditation checklist standards. On the other hand, PEP 
provides specific feedback of a formative nature that helps administrators 
implement meaningful, in-process changes. 

Fifth, PEP serves valuable in-service and consultation functions. 
The peer team that is visiting the other IU will see a wide diversity of 
strong and weak services in operation, The alert team member will not only 
take note of strengths and weaknesses for the purpose of the people whose 
programs are being evaluated, but will also take mental note of how such 
ideas can be used back in their own IU. Similarly, the IU whose programs 
are. being evaluated can look upon the peer evaluative findings as truly, 
substantial consultation coming from people who can identify with the 
specific types of problems of the local IU. 

Sixth, there is participation of all staff members at the local level. 
In Che beginning of the year teachers are asked to examine areas of strength 
or weaknesses and make specific recommendations. Feedback becomes part 
of the self ^evaluation process involving supervisory and other local 
administrative staff* Thus, all local staff have had a chance to identify 
with the programs. 

Seventh, many changes can be identified that require no increase 
in human or non-human resources. Planning viewed as a process directs 
attention to the activities in which humans engage themselves, PEP high- 
lights the concept of co-mingling resources by focusing on the individual 
needs of exceptional children and youth and delineates who and how best 
the resources can improve each individual's planned special education 
intervention progression. 



Expanding PEP 



PEP will be implemented to evaluate all intermediate unit special 
education programs during the 1973-1974 school year. 

During the 1973-1974 school year, a pilot project using PEP will be 
initiated in school districts that operate special education programs. 
This pilot project will field test the PEP approach on another level; 
the school district operation of special education programs and services. 

It is felt by the staff that through developing PEP, this systematic 
peer evaluation program can have a wide ranging effect on evaluating any 
educational program. It is hoped that PEP will be used to evaluate 
general education programs in the near future. The PEP staff will work 
with other agencies in planning and implementing PEP into general 
program operations. 



PEER EVALUATION PROGRAM 



1973-1974 
GENERAL PROCEDURES 

Pre-Visit Preparations 

A, Internal Evaluation (Self-Review) - all intermediate units will 
be required to develop self -review protocol which will enable 

the Local Education Agency staff to determine their own individual 
strengths and weaknesses and to provide relevant feedback con- 
cerning recommended change where needed, 

B, Selection of Evaluators (Self-Review) - all currently employed 
special education staff members will be asked to participate in 
the internal (self -review) evaluation phase of this program. 
Criteria will be devleoped and can be found in the Teacher-Feedback 
section of this procedures manual* 

C, Requested Time Lines (Self -Review) internal evaluations are to 
commence after the third week of the school year beginning in 
Spetember and should be concluded by November 15, 



D. In-Service Programs (Self -Review) - it is recommended that the 
special education director, acting as Evaluation Chairman, shall 
provide appropriate, in-service activities designed to acquaint 
all staff with the evaluation protocol to be used, etc. 

E. Internal Reports (Self -Review) - between November 15 and December 
14, all internal repot cs are to be written, completed and submitted 
to the Chairman for review and compilation. The finalized document 
shall then be submitted to the Bureau of Special and Compensatory 
Education, Division of Special Education* Also > each special 
education staff member employed by the LEA shall receive a copy 

of this report, provided by the intermediate unit. 

F. Selection of Evaluation Team Member s (External Evaluation) - the 
special education director shall select team members who will 
externally evaluate ccher special education programs, i.e., pro- 
grams from surrounding or nearby intermediate units • Procedures 
for selecting these team memb *vs cm be found in the "Selection of 
Team Members 11 section of this report. 

Arrangements for On-site Visits (External Evaluation) 
A. The Chairman will contact the intermediate unit special education 
director whose program is to be evaluated and shall request: 

1. Dates for external evaluation (should take place between 
January 15 and March 1) 

2. Intermediate unit Special Education Plan 

3. Areas of Special Education programs that need "special attention 1 ' 
from evaluators 



B. The intermediate unit special education director shall inform his 
executive director, board, news media and other appropriate 
dissemination source3 of the on-site visit. This information shall 
contain the purpose of the visit and what its impact will have on 
the Special Education program. All information should be provided 
at least two (2) weeks from the actual on-site visit, 

C. The intermediate unit special education director shall notify, by 
let tar, the intermediate unit to be evaluated. This shall include 
the dates of the evaluation, name of committee members and final 
confirmation related to: 

1 . Accommodations 

2, Hospitality arrangements, etc. 

III. On-site visits (January 15 - March 1) - The following procedures identify 
the mechanics of the on-site evaluation ?:eam, 

A. Arrival - All team members shall arrive the night before. A general 
orientation meeting shall be scheduled (7:00 PM) by the local special 
education director with the evaluation team. The local special 
education director shall: 

1. Bring to the attention of the team those areas of the program 
that have been specifically identified as "weak" by his own staff, 
or Pennsylvania Department of Education (Division of Special 
Education) 

2. Review hy mutual agreement, how the agenda shall be followed 
during the three day visit. 

B. Procedures 

1. 1st day (AM) - detailed discussion with each supervisor about his 

or her program. 



2. 1st Day (PM) - actual visitation of programs 

3. 2nd Day (AM-PM) - continued program evaluations 

4. 3rd Day (AM) - each team member prepares informal evaluation 

notes to be used as guides to presentation 
before local staff members, 

(AM) Discuss vith Chairman how best to present 
findings to local staff, 

(AM) Recommendations shall be made by Chairman to the 
team members to identify exemplary areas of the 
program. However, the local point of each presen- 
tation should identify weaknesses with specific 
recommendations for improvement, 

(PM) Luncheon - formal verbal presentation of findings, 
C. Post-Visit Feedback 

1. The Chairman shall request indivi'ial team members to submit, in 
writing an evaluation report within ten (10) days, 

2. Twenty (20) days after receiving these reports the Chairman shall 
submit the compiled findings (original and five copies) to the 
director of the Bureau of Special and Compensatory Education. 

IV • Change Mechanism for Implementing Recommenda t ions 
A. Local team 

1. Analyze all feedback (self-evaluation reports compared with peer 
evaluation reports) and other evaluative reports, e.g., Federal 
Title III Program. 

2. Agrees upon a strategy for change. This means: 

a. The identification of appropriate area for change 

b. Procedures for implementing change in each area 



c. Analysis of implications for overall intermediate unit 
functioning when a particular procedure is implemented 
under any given area of change. 

d. Under any given area of change there shall be a selection 
of the most feasible alternative for implementing the 
identified area of change, 

Final Disposition of M Change Strategies 1 ' 

The Chairman discusses . recommended change with the executive director 
prior to presentation to the Intermediate Unit Board. The executive 
director and chairman must agree upon the strategies for change and 
write amendments to the Intermediate Unit Plan prior to submission 
to the intermediate unit board for acceptance. The executive director 
and/or special education director (protocol) present amendments to 
intermediate unit board for approval. Appropriate resolutions shall be 
made and be submitted to the, Division of Special Education as part of 
the intermediate unit plan. Resolution shall be submitted in accordance 
with Department of Education policies (July 1) as part of its dissem- 
ination procedures. The local intermediate should submit the final 
results of the evaluation to the local news media for immediate release. 
Implementation and Monitoring of Board Resolutions 

The local staff shall be directly involved in the implementation and 
monitoring of the board resolution which were the result cf the 
evaluation team's activities. Major steps shall be identified and 
agreed upon that will assure the implementation of resolutions and 
provide built-in "local" monitoring system. 

Division of Special Education staff personnel shall, during the 
ensuing school year, closely monitor these changes which were recommended 
for implementation. An important aspect of this monitoring will be 
the new cycle of peer evaluations. 



Selection of Team Members 

The most important single component of the peer evaluation program, 
(PEP), is the selection of the evaluation team* Usually, the chairman will 
be the intermediate unit director of special education. In most cases the 
team members selected will be supervisors, instructional advisers, or master 
teachers because of their expertise and their programwide knowledge of 
intermediate unit operations. The chairman shall make all team member 
selections by keeping the following criteria in mind: 

A. Amount of practical experience the potential team member has 
had in the areas of supervision and instruction in fields of 
exceptionality. 

B. Amount of theoretical background, current awareness and interest, 
and level-headed analytical skills the staff member displays with 
regard to the fields of exceptionality. 

C. Personal qualities that are conducive to working as part of a team, 
to presenting sometimes delicate findings in an inoffensive yet 
honest and constructively critical fashion, and other interpersonal 
qualities. 

D. Ability to objectively "evaluate" total programs , not only from 
teacher level, but more as a component of the whole intermediate unit 
plan. 

The chairman of the team must look at the selection in two ways. First , 
he must look for the areas in the intermediate unit program that he feels 
represent the bulk of operations in the intermediate unit and select those 
people that have the responsibility of supervising those areas. This is the 
evaluation component. Second , he must look at the areas of the program that he 



believes need change and select one or more people most appropriate to 
looking at a program that apparently is weak. 

The following is a suggested team that may be involved in PEP. This 
list is intended to be a guideline in the process of selecting a team, and 
some of the members and grouping may not be applicable to a given intermediate 
unit. However, each of the major program areas of an intermediate should be 
included in team selection. Each team should consist of from six to eight 
members and should include the following areas: 

1. Chairman (one member) 1 

2. General administrator from intermediate unit (executive director, 
assistant executive director) 

3. Psychologist (one member) 

4. Mental retardation (one member) 

5. Speech and hearing impairment and/ur visual impairment (one 
member) 

6. Social and emotional disturbance and/or brain injury/learning 
disabled (one member) 

7. Approved private school representation (one member) 

8-9. Physical handicaps and/or gifted and/or multiple handicaps and/or 
detention homes and/or other category (two members) 
Special attention should be paid to the selection of the approved private 
school representative. The chairman shall select an approved private school 
representative as a member~at-large. This person should be a key administrator 
of an approved private school so that he or she will have an overall picture of the 
special education operations and can lend expertise in program planning and 
change. (The private school member is used only during the on-site visit, not the 
self-evaluation). The private school member will serve in his or her member- 



at-Iarge role by paying particular attention to the details of the total 
intermediate unit plan presented to team members on the first day of the 
on-site visit. 

The chairman and psychologist member make reactions upon the same member- 
at-large basis. Thus, in effect, there are 3 members-at-large. During the 
first and second days when visits to classrooms are in progress, the 3 
members-at-large may accompany any other team member whose specific area of 
program responsibility is of interest* 

Finally, it is strongly suggested that a Division of Special Education 
staff person be an observer during the evaluation visit* This person will 
only be an observer-at-large and will not have any input in regard to the 
evaluation process. 
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PEER EVALUATION PROGRAM 
1973-1974 

TEACHER INPUT INSTRUMENT FOR SELF-EVALUATION 



Bureau of Special and Compensatory Education 
Division of Special Education 

Instructions: During 1973-1974, your local intermediate unit is working with 
the Division of Special Education in Harrisburg to bring about systematic 
improvements in each of the 29 Intermediate Unit Special Education program 
operations. Each intermediate unit has been paired with another intermediate 
unit. During the first half of the year, each intermediate unit will conduct 
a self -evaluation of its own programs for exceptional children. This "Teacher 
Input 11 form is a crucial part of"' the self -evaluation phase. During the 
second half of the year, the two intermediate unit's in 'each pairing will 
exchange small evaluation teams to obtain an objective "outside" appraisal 
of strengths and weaknesses of intermediate unit programs. 

From the perspective of your day-to-day classroom operations, please list 
strengths, weaknesses, and specific recommendations for change for each of 
the following. Write on back of form if more space is required or attach 
additional sheets. Since your name is not required, we want frank criticism. 
Please mail the form, aftar time for careful consideration, to your intermediate 
unit director of special education. 

1. Facilities (e.g., space, lighting, ventilation) 

a. Strengths: - 



b. Weaknesses: 



c. Specific recommendations for change: 



2. Materials and equipment (i.e., adequate supply, easy access to those 
not housed in room, appropriateness to curricular content areas and 
nature of children) : 
a. Strength: 



b. Weaknesses: 



c. Specific recommendations for change: 



3. Supervisory support (i.e., appear on regular basis, make specific 

recommendations that seem appropriate to limits and strengths of the 
classroom setting, coordinate support system with various other staff 
q membero who might be helpful to a child *s problems: 

ERIC 



b. Weaknesses : 



c, Specific recommendations for change: 



Specialized support services, such as psychologists, therapists, and 
social workers (i.e.,- psychologist makes specific programming recommendations 
for a child in the evaluation report that are appropriate for the child's 
problems and are suitable for the available resources; therapists attempt 
to coordinate their services with the schedule of the teacher): 

a. Strengths: 



b. Weaknesses: 



c. Specific recommendations for change; 



5. Curricular considerations (i.e., specific, useful, and structured 
curriculum guide is available for your area of exceptionality from 
the intermediate unit; attempts are made to coordinate your curriculum 
with the sending or receiving school); 
a. Strengths: 



b. Weaknesses: 



c. Specific recommendations for change: 



6. General comments and suggestions for change: 



PEER EVALUATION PROGRAM 
1973-1974 



CRITERIA FOR BOTH SELF - AND PEER EVALUATIONS 
Bureau of Special and Compensatory Education 
Division of Special Education 

Directions: This form is designed to be used only by the intermediate unit 
evaluation teams (6 to 8 members) that have been paired in the Peer Evaluation 
Program (PEP). The procedures outlined in the August, 1973, guidelines for 
PEP should be followed. When this form is used in the self -evaluation phase, 
Part I required the most work, since Part II can be completed by synthesizing 
the major findings from the start-of -the-year "Teacher Input for Self- 
Evaluation 11 form. When the "Criteria" form is used in a peer evaluation, 
then both Part I and Part II must be completed at that time from fresh inter- 
views. Note that Items I through N of Part I are the same as the items under 
Part II; it will be of interest to note the discrepancy on these criteria 
between administrative and supervisory personnel on the one hand and teaching 
staff on the other hand. 

Each intermediate unit evaluation team member is to complete one of these 
"Criteria" forms, (please use back of form or attach additional sheets as 
necessary). So that the chairman may get back to an individual team member at 
a later time with questions regarding compilation into a single, final report, 
each team member is requested to list his or her name and area of examination: 

Name: 

Area: 



I. Visits with administrative and supervisory personnel 

A. Intermediate unit plan (i.e., philosophy, inclusion of all crucial 

components, how many staff members had input into the plan, specificity, 
objectives, philosophy) : 

Strengths: ^ 



Weaknesses : 



Specific recommendations for change: 



B. Professional, non-teaching staff deployment (i.e., use of psychologists 
time, duties of supervisors, gaining maximum benefits from itinerant 
services) : 

Strengths; ' __ 



Weaknesses : 



Specific recommendations for change: 



Paraprof essional staff deployment (i.e., teacher aides, volunteers, 
secretaries) : 

Strengths; ■ 



Weaknesses : 



Specific recommendations for change: 



Pupil referral and placement procedures (i.e., inappropriate referrals 
follow-up on referrals, turnaround time till placement, inappropriate 
placements) : 

Strengths: 



Weaknesses: 



Specific recommendations for change: 



Overall program cfesign (i.e., adequacy of interrelationship among 
programs for various exceptionality areas: adequacy of interrelation- 
ships among functions provided under self-contained classes, resource 
rooms, itinerant services, and home services): 

Strengths: ^ 



Weaknesses: 



Specific recommendations for change: 



Involvement: with local districts (i.e., planning, reintegration of 
children with regular classes, intermediate unit supervisory relation- 
ship with district-operated classes, rendering of intermediate unit 



psychological services, joint in-service arrangements); 
Strengths: 



Weaknesses : 



G, Relationships and coordination with private schools and other agencies 
(i.e., appropriate use of their capabilities, planning): 
Strengths: 



Weaknesses: 



Specific recommendations for change: 



H. Staff qualifications (i.e., certification, experience): 



Weaknesses: 



Specific recommendations for change; 



I. Facilities (i.e.* space, lighting, ventilation): 
Strengths: 



Weaknesses: 



Specific recommendations for change: 



J. Materials and equipment (i.e., adequate supply, easy access to those 
not housed in room, appropriateness to curricular content areas and 
nature of children): 



Weaknesses : 



Specific recommendations for change; 



K. Supervisory support (i.e., appear on regular basis, make specific 

recommendations that seem appropriate to limits and strengths of the 
classroom setting, coordinate support system with various other staff 
members who might be helpful to a child's problems: 

Strengths: 



Weaknesses: 



Specific recommendations for change: 



L. Specialized support services, such as psychologists, therapists, and social 
workers (i.e., psychologist makes specific programming recommendations for 



a cMld in the evaluation report that are appropriate for the child's problems 
and are suitable for the available resources; therapists attempt to 
coordinate their services with the schedule of the teacher): 
Strengths: 



Weaknesses : 



Specific recommendations for change: 



M. Curricular consideration (i.e., a specific, useful and structured curriculum 
guide is available for your area of exceptionality from the intermediate 
unit; attempts are made to coordinate your curriculum with the sending or 
receiving school); 

Strengths: ^ 



Weaknesses : 



Specific recommendations for change: 



No Genera" 1 comments and suggestions for change: 



II. Visits with teaching staff and examination of actual programs 
A. Facilities (i.e., space, lighting, ventilation): 
Strengths: 



Weaknesses : 



Specific recommendations for change; 



Materials and equipment (i.e», adequate supply, easy access to those 
not housed in room, appropriateness to curricular content areas and 
nature of children) : 

Strengths: _____ 



Weaknesses : 



Specific recommendations for change: 



Supervisory support (i.e., appear on regular bases, make specific 
recommendations that seem appropriate to limits and strengths of the 
classroom setting, coordinate support system with various other staff 
members who might be helpful to a child T s problems): 
Strengths: 



Weaknesses: 



Specific recommendations for change: 



Specialized support services, such as psychologists, therapists, and 
social workers (i.e., psychologist makes specific programming 
recommendations for a child in the evaluation report that are appro- 
priate for the child's problems and are suitable for the available 
resources; cherapists attempt to coordinate their services with the 
schedule of the teacher): 

Strengths: 



Weaknesses: 



Specific recommendations for change: 



Curricular considerations (i.e., a specific, useful, anc 1 structured 
curriculum guide is available for your area of exceptionality from the 
intermediate unit; attempts are made to coordinate your curriculum with 
the sending or receiving school) : 



Strengths: 



Weaknesses : 



Specific recommendations for change: 



F. General comments and suggestions for change: 
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